Generation of Artificial Image and Video Data for Medical Deep Learning Applications

Neuronale Netze haben in den letzten Jahren erstaunliche Ergebnisse bei der Erkennung von Ereignissen im Bereich der medizinischen Bild- und Videoanalyse erzielt. Dabei stellte sich jedoch immer wieder heraus, dass ein genereller Mangel an Daten besteht. Dieser Mangel bezieht sich nicht nur auf die Anzahl an verfügbaren Datensätzen, sondern auch auf die Anzahl an individuellen Stichproben, das heißt an unabhängigen Bildern und Videos, in bestehenden Datensätzen. Das führt wiederum zu einer schlechteren Erkennungsgenauigkeit von Ereignissen durch das neuronale Netz. Gerade im medizinischen Bereich ist es nicht einfach möglich die Datensätze zu erweitern oder neue Datensätze zu erfassen. Die Gründe hierfür sind vielfältig. Einerseits können rechtliche Belange die Datenveröffentlichung verhindern. Andererseits kann es sein, dass eine Krankheit nur sehr selten Auftritt und sich so keine Gelegenheit bietet die Daten zu erfassen. Ein zusätzliches Problem ist, dass es sich bei den Daten meist um eine sehr spezifische Domäne handelt, wodurch die Daten meist nur von Experten annotiert werden können. Die Annotation ist aber zeitaufwendig und somit teuer. Existierende Datenaugmentierungsmethoden können oft nur sinnvoll auf Bilddaten angewendet werden und erzeugen z.B. bei Videos nicht ausreichend zeitlich unabhängige Daten. Deswegen ist es notwendig, dass neue Methoden entwickelt werden, mit denen im Nachhinein auch Videodatensätze erweitert oder auch synthetische Daten generiert werden können. Im Rahmen dieser Dissertation werden zwei neu entwickelte Methoden vorgestellt und beispielhaft auf drei medizinische Beispiele aus dem Bereich der Chirurgie angewendet. Die erste Methode ist die sogenannte Workflow-Augmentierungsmethode, mit deren Hilfe semantischen Information, z.B. Ereignissen eines chirurgischen Arbeitsablaufs, in einem Video augmentiert werden können. Die Methode ermöglicht zusätzlich auch eine Balancierung zum Beispiel von chirurgischen Phasen oder chirurgischen Instrumenten, die im Videodatensatz vorkommen. Bei der Anwendung der Methode auf die zwei verschiedenen Datensätzen, von Kataraktoperationen und laparoskopischen Cholezystektomieoperationen, konnte die Leistungsfähigkeit der Methode gezeigt werden. Dabei wurde Genauigkeit der Instrumentenerkennung bei der Kataraktoperation durch ein Neuronales Netz während Kataraktoperation um 2,8% auf 93,5% im Vergleich zu etablierten Methoden gesteigert. Bei der chirurgischen Phasenerkennung im Fall bei der Cholezystektomie konnte sogar eine Steigerung der Genauigkeit um 8,7% auf 96,96% im Verglich zu einer früheren Studie erreicht werden. Beide Studien zeigen eindrucksvoll das Potential der Workflow-Augmentierungsmethode. Die zweite vorgestellte Methode basiert auf einem erzeugenden gegnerischen Netzwerk (engl. generative adversarial network (GAN)). Dieser Ansatz ist sehr vielversprechend, wenn nur sehr wenige Daten oder Datensätze vorhanden sind. Dabei werden mit Hilfe eines neuronalen Netzes neue fotorealistische Bilder generiert. Im Rahmen dieser Dissertation wird ein sogenanntes zyklisches erzeugendes gegnerisches Netzwerk (engl. cycle generative adversarial network (CycleGAN)) verwendet. CycleGANs führen meiste eine Bild zu Bild Transformation durch. Zusätzlich ist es möglich weitere Bedingungen an die Transformation zu knüpfen. Das CycleGAN wurde im dritten Beispiel dazu verwendet, ein Passbild von einem Patienten nach einem Kranio-Maxillofazialen chirurgischen Korrektur, mit Hilfe eines präoperativen Porträtfotos und der operativen 3D Planungsmaske, zu schätzen. Dabei konnten realistisch, lebendig aussehende Bilder generiert werden, ohne dass für das Training des GANs medizinische Daten verwendeten wurden. Stattdessen wurden für das Training synthetisch erzeugte Daten verwendet. Abschließend lässt sich sagen, dass die in dieser Arbeit entwickelten Methoden in der Lage sind, den Mangel an Stichproben und Datensätzen teilweise zu überwinden und dadurch eine bessere Erkennungsleistung von neuronalen Netzen erreicht werden konnte. Die entwickelten Methoden können in Zukunft dazu verwendet werden, bessere medizinische Unterstützungssysteme basierende auf künstlicher Intelligenz zu entwerfen, die den Arzt in der klinischen Routine weiter unterstützen, z.B. bei der Diagnose, der Therapie oder bei bildgesteuerten Eingriffen, was zu einer Verringerung der klinischen Arbeitsbelastung und damit zu einer Verbesserung der Patientensicherheit führt

Coding over Sets for DNA Storage

In this paper, we study error-correcting codes for the storage of data in synthetic deoxyribonucleic acid (DNA). We investigate a storage model where data is represented by an unordered set of $M$ sequences, each of length $L$. Errors within that model are losses of whole sequences and point errors inside the sequences, such as substitutions, insertions and deletions. We propose code constructions which can correct these errors with efficient encoders and decoders. By deriving upper bounds on the cardinalities of these codes using sphere packing arguments, we show that many of our codes are close to optimal.Comment: 5 page

Simulation-based estimation of the number of cameras required for 3D reconstruction in a narrow-baseline multi-camera setup

Graphical visualization systems are a common clinical tool for displaying digital images and three-dimensional volumetric data. These systems provide a broad spectrum of information to support physicians in their clinical routine. For example, the field of radiology enjoys unrestricted options for interaction with the data, since information is pre-recorded and available entirely in digital form. However, some fields, such as microsurgery, do not benefit from this yet. Microscopes, endoscopes, and laparoscopes show the surgical site as it is. To allow free data manipulation and information fusion, 3D digitization of surgical sites is required. We aimed to find the number of cameras needed to add this functionality to surgical microscopes. For this, we performed in silico simulations of the 3D reconstruction of representative models of microsurgical sites with different numbers of cameras in narrow-baseline setups. Our results show that eight independent camera views are preferable, while at least four are necessary for a digital surgical site. In most cases, eight cameras allow the reconstruction of over 99% of the visible part. With four cameras, still over 95% can be achieved. This answers one of the key questions for the development of a prototype microscope. In future, such a system can provide functionality which is unattainable today

Design of an experimental four-camera setup for enhanced 3D surface reconstruction in microsurgery

Future fully digital surgical visualization systems enable a wide range of new options. Caused by optomechanical limitations a main disadvantage of today’s surgical microscopes is their incapability of providing arbitrary perspectives to more than two observers. In a fully digital microscopic system, multiple arbitrary views can be generated from a 3D reconstruction. Modern surgical microscopes allow replacing the eyepieces by cameras in order to record stereoscopic videos. A reconstruction from these videos can only contain the amount of detail the recording camera system gathers from the scene. Therefore, covered surfaces can result in a faulty reconstruction for deviating stereoscopic perspectives. By adding cameras recording the object from different angles, additional information of the scene is acquired, allowing to improve the reconstruction. Our approach is to use a fixed four-camera setup as a front-end system to capture enhanced 3D topography of a pseudo-surgical scene. This experimental setup would provide images for the reconstruction algorithms and generation of multiple observing stereo perspectives. The concept of the designed setup is based on the common main objective (CMO) principle of current surgical microscopes. These systems are well established and optically mature. Furthermore, the CMO principle allows a more compact design and a lowered effort in calibration than cameras with separate optics. Behind the CMO four pupils separate the four channels which are recorded by one camera each. The designed system captures an area of approximately 28mm × 28mm with four cameras. Thus, allowing to process images of 6 different stereo perspectives. In order to verify the setup, it is modelled in silico. It can be used in further studies to test algorithms for 3D reconstruction from up to four perspectives and provide information about the impact of additionally recorded perspectives on the enhancement of a reconstruction

Electrocardiographic Imaging Using a Spatio-Temporal Basis of Body Surface Potentials - Application to Atrial Ectopic Activity

Electrocardiographic imaging (ECGI) strongly relies on a priori assumptions and additional information to overcome ill-posedness. The major challenge of obtaining good reconstructions consists in finding ways to add information that effectively restricts the solution space without violating properties of the sought solution. In this work, we attempt to address this problem by constructing a spatio-temporal basis of body surface potentials (BSP) from simulations of many focal excitations. Measured BSPs are projected onto this basis and reconstructions are expressed as linear combinations of corresponding transmembrane voltage (TMV) basis vectors. The novel method was applied to simulations of 100 atrial ectopic foci with three different conduction velocities. Three signal-to-noise ratios (SNR) and bases of six different temporal lengths were considered. Reconstruction quality was evaluated using the spatial correlation coefficient of TMVs as well as estimated local activation times (LAT). The focus localization error was assessed by computing the geodesic distance between true and reconstructed foci. Compared with an optimally parameterized Tikhonov-Greensite method, the BSP basis reconstruction increased the mean TMV correlation by up to 22, 24, and 32% for an SNR of 40, 20, and 0 dB, respectively. Mean LAT correlation could be improved by up to 5, 7, and 19% for the three SNRs. For 0 dB, the average localization error could be halved from 15.8 to 7.9 mm. For the largest basis length, the localization error was always below 34 mm. In conclusion, the new method improved reconstructions of atrial ectopic activity especially for low SNRs. Localization of ectopic foci turned out to be more robust and more accurate. Preliminary experiments indicate that the basis generalizes to some extent from the training data and may even be applied for reconstruction of non-ectopic activity

The Impact of Standard Ablation Strategies for Atrial Fibrillation on Cardiovascular Performance in a Four-chamber Heart Model

Atrial fibrillation is one of the most frequent cardiac arrhythmias in the industrialized world and ablation therapy is the method of choice for many patients. However, ablation scars alter the electrophysiological activation and the mechanical behavior of the affected atria. Different ablation strategies with the aim to terminate atrial fibrillation and prevent its recurrence exist but their impact on the hemodynamic performance of the heart has not been investigated thoroughly. In this work, we present a simulation study analyzing five commonly used ablation scar patterns and their combinations in the left atrium regarding their impact on the pumping function of the heart using an electromechanical whole-heart model. We analyzed how the altered atrial activation and increased stiffness due to the ablation scar affect atrial as well as ventricular contraction and relaxation. We found that systolic and diastolic function of the left atrium is impaired by ablation scars and that the reduction of atrial stroke volume of up to 11.43% depends linearly on the amount of inactivated tissue. Consequently, the end-diastolic volume of the left ventricle, and thus stroke volume, was reduced by up to 1.4% and 1.8%, respectively. During ventricular systole, left atrial pressure was increased by up to 20% due to changes in the atrial activation sequence and the stiffening of scar tissue. This study provides biomechanical evidence that atrial ablation has acute effects not only on atrial contraction but also on ventricular pumping function. Our results have the potential to help tailoring ablation strategies towards minimal global hemodynamic impairment

The Impact of Standard Ablation Strategies for Atrial Fibrillation on Cardiovascular Performance in a Four-Chamber Heart Model

Purpose: Atrial fibrillation is one of the most frequent cardiac arrhythmias in the industrialized world and ablation therapy is the method of choice for many patients. However, ablation scars alter the electrophysiological activation and the mechanical behavior of the affected atria. Different ablation strategies with the aim to terminate atrial fibrillation and prevent its recurrence exist but their impact on the performance of the heart is often neglected. Methods: In this work, we present a simulation study analyzing five commonly used ablation scar patterns and their combinations in the left atrium regarding their impact on the pumping function of the heart using an electromechanical whole-heart model. We analyzed how the altered atrial activation and increased stiffness due to the ablation scars affect atrial as well as ventricular contraction and relaxation. Results: We found that systolic and diastolic function of the left atrium is impaired by ablation scars and that the reduction of atrial stroke volume of up to 11.43% depends linearly on the amount of inactivated tissue. Consequently, the end-diastolic volume of the left ventricle, and thus stroke volume, was reduced by up to 1.4 and 1.8%, respectively. During ventricular systole, left atrial pressure was increased by up to 20% due to changes in the atrial activation sequence and the stiffening of scar tissue. Conclusion: This study provides biomechanical evidence that atrial ablation has acute effects not only on atrial contraction but also on ventricular performance. Therefore, the position and extent of ablation scars is not only important for the termination of arrhythmias but is also determining long-term pumping efficiency. If confirmed in larger cohorts, these results have the potential to help tailoring ablation strategies towards minimal global cardiovascular impairment

The impact of standard ablation strategies for atrial fibrillation on cardiovascular performance in a four-chamber heart model

Purpose: Atrial fibrillation is one of the most frequent cardiac arrhythmias in the industrialized world and ablation therapy is the method of choice for many patients. However, ablation scars alter the electrophysiological activation and the mechanical behavior of the affected atria. Different ablation strategies with the aim to terminate atrial fibrillation and prevent its recurrence exist but their impact on the hemodynamic performance of the heart has not been investigated thoroughly. Methods: In this work, we present a simulation study analyzing five commonly used ablation scar patterns and their combinations in the left atrium regarding their impact on the pumping function of the heart using an electromechanical whole-heart model. We analyzed how the altered atrial activation and increased stiffness due to the ablation scar affect atrial as well as ventricular contraction and relaxation. Results: We found that systolic and diastolic function of the left atrium is impaired by ablation scars and that the reduction of atrial stroke volume of up to 11.43% depends linearly on the amount of inactivated tissue. Consequently, the end-diastolic volume of the left ventricle, and thus stroke volume, was reduced by up to 1.4% and 1.8%, espectively. During ventricular systole, left atrial pressure was increased by up to 20% due to changes in the atrial activation sequence and the stiffening of scar tissue. Conclusion: This study provides biomechanical evidence that atrial ablation has acute effects not only on atrial contraction but also on ventricular pumping function. Our results have the potential to help tailoring ablation strategies towards minimal global hemodynamic impairment